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Abstract 

Few standardized tools are available for time-efficient screening of emotional 
health status across diagnostic categories, especially in primary care. We evalu- 
ated the 45-question Brief Risk-resilience Index for Screening (BRISC) and the 
15-question mini-BRISC in identifying poor emotional health and coping 
capacity across a range of diagnostic groups - compared with a detailed clinical 
assessment - in a large sample of adult outpatients. Participants 18-60 years of 
age (n = 1079) recruited from 12 medical research and clinical sites completed 
the computerized assessments. Three index scores were derived from the full 
BRISC and the mini-BRISC: one for risk (negativity-positivity bias) and two 
for coping (resilience and social capacity). Summed answers were converted to 
standardized z-scores. BRISC scores were compared with detailed health assess- 
ment and diagnostic interview (for current psychiatric, psychological, and neu- 
rological conditions) by clinicians at each site according to diagnostic criteria. 
Clinicians were blinded to BRISC scores. Clinical assessment stratified partici- 
pants as having "clinical" (n = 435) or "healthy" (« = 644) diagnostic status. 
Receiver operating characteristic analyses showed that a z-score threshold of 
— 1.57 on the full BRISC index of emotional health provided an optimal classi- 
fication of "clinical" versus "healthy" status (sensitivity: 81.2%, specificity: 
92.7%, positive predictive power: 80.2%, and negative predictive power: 
93.1%). Comparable findings were revealed for the mini-BRISC. Negativity- 
positivity bias index scores contributed the most to prediction. The negativity- 
positivity index of emotional health was most sensitive to classifying major 
depressive disorder (100%), posttraumatic stress disorder (95.8%), and panic 
disorder (88.7%). The BRISC and mini-BRISC both offer a brief, clinically use- 
ful screen to identify individuals at risk of disorders characterized by poor emo- 
tion regulation, from those with good emotional health and coping. 



Introduction 

Emotional dysregulation is a feature of multiple psychiat- 
ric, psychological, and neurological conditions, and con- 
versely, effective emotional regulation characterizes 
positive well-being, coping, and resilience. Our aim was 
to use these features to identify a broad screen for poor 



versus good emotional health across diagnostic and com- 
munity samples. 

Approximately 60% of patients who have psychiatric 
and neurological disorders seek care from primary care 
physicians (Regier et al. 1978; Ezzati-Rice and Rohde 
2008). Clinicians who are not psychiatric or neurological 
specialists are increasingly expected to serve roles in early 
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identification, management, and ultimately prevention of 
these disorders. (Druss et al. 2010). To support these 
roles, there is demand for a quick screen that can be 
applied across broad populations and provide immediate 
feedback. Ideally, such screening tools would be time 
effective for both physician - given typical heavy patient 
loads - and patient - picking up a broad set of conditions 
earlier and more effectively. They would provide an 
objective and accurate way to identify individuals at risk 
of psychiatric and neurological conditions, and factor in 
behaviors which contribute to resilience and capacity to 
cope. Furthermore, they would provide immediate feed- 
back on case identification via automated reporting. 

There is currently a dearth of standardized tools that 
provide a broad screen of this kind. At the population 
level, mental health-related disorders go unidentified and 
thus untreated in 50-65% of cases (Nielson and Williams 
1980; Kessler et al. 1985; Schulberg et al. 1985; Katon 
1987; Barret et al. 1988; Bonis et al. 1988; Schulberg and 
Burns 1988; Andersen and Harthorn 1989; Ormel et al. 
1991; Rydon et al. 1992). Of the available self-report 
screening scales that could be considered brief and com- 
prising sound psychometric properties, the focus is on 
screening for a particular diagnosis (Mulrow et al. 1995). 
For example, the Patient Health Questionnaire-9 item 
(PHQ-9) screens specifically for diagnostic criteria of 
depressive disorder (Kroenke et al. 2010), and the 
Quick Inventory of Depressive Symptoms - Self-Report 
(QIDS-SR) assesses the severity of symptoms in major 
depressive disorder (Rush et al. 2003). Other scales are 
focused on health-related outcomes. For example, the 
Medical Outcomes Study Short Form (SF-36; Ware and 
Sherbourne 1992) and its even shorter version (SF-12) are 
a psychometrically sound survey designed to assess quality- 
of-life outcomes across diagnoses. It is not intended as a 
screening tool. Other pan-diagnostic scales with robust 
psychometric qualities are focused on outcomes for a 
related set of diagnoses. For example, the OASIS is a brief 
self-report scale for assessing frequency of anxiety, intensity 
of anxiety, behavioral avoidance, and functional impair- 
ment associated with anxiety to determine symptom and 
functional outcomes across diagnoses of anxiety disorder 
(Campbell-Sills et al. 2009). 

The BRISC is designed to address gaps in these avail- 
able tools. First, it provides a quick screen for emotional 
health relative to a wide spectrum of diagnoses and 
healthy people, which is not available in currently avail- 
able instruments. This enables identification of cases at 
risk of poor mental and neurological health across various 
disorders and practice settings. Second, it includes mea- 
sures of coping to inform the triage of those most at risk 
and coping poorly versus those who are resilient and 
coping well. This information is also not provided by 
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available instruments. The BRISC has been validated 
against other self-report measures of emotional health, 
functional outcome measures, and biological susceptibility 
factors (for details, see Methods). It is designed to pro- 
vide a time- and cost-effective screen, delivered via the 
web, with immediate reporting on results. 

This study was designed to evaluate the sensitivity, 
specificity, and predictive power of the 45-item BRISC 
and the 15-item "mini-BRISC" in distinguishing clinical 
versus healthy status across a range of disorders in a large 
sample of adult outpatients and healthy volunteers. 
BRISC scores were compared with a detailed assessment 
of clinical status. 

Method 
The BRISC 

The BRISC was developed and validated within a frame- 
work called the "INTEGRATE model", which draws on 
psychiatric, psychological, physiological, and neuroscience 
theories (Gordon et al. 2008; Williams et al. 2008). It is 
designed to measure, by self-report, the spectrum of good 
versus poor self-regulation of emotional functions, which 
underlies mental health and has a basis in neurobiology. 

The BRISC measures three core domains: negativity 
bias, emotional resilience, and social skills. Negativity bias 
represents hypersensitivity to stress and the expectation of 
negative outcomes, which elevate the risk for poor brain 
health (Wichers et al. 2007; Williams et al. 2009, 2010). 
Positivity Bias is the opposing tendency and quantifies a 
lack of negativity bias and an expectation of positive and/ 
or neutral outcomes. Emotional resilience is the capacity 
for self-efficacy. It is premised in the notion that having a 
"thick skin" (or emotional resilience) may offset poor 
mental functioning and facilitate good functioning. Social 
skills is the capacity to engage socially and seek support. 
These attributes contribute to the ability to cope with 
poor mental functioning and to facilitate good function- 
ing. Development of the BRISC followed a stepwise pro- 
cess which is detailed in its manual (Brain Resource Ltd 
publishers 2010). The five main validation steps are sum- 
marized below: 

Construct validation of content domains 

These three domains were validated by principal compo- 
nents analyses of an initial pool of 93 items (Rowe et al. 
2007; Williams et al. 2008). Factor analysis confirmed the 
presence of domains reflecting negativity-positivity bias, 
emotional resilience, and social skill capacity in a healthy 
volunteer sample of 1000 individuals who spanned nine 
decades in age (Rowe et al. 2007; Williams et al. 2008). 
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Using regression analysis, we reduced the number of 
items loading on these factors to the core 45 items needed 
to predict them (Wichers et al. 2007). This structure was 
replicated in an independent sample of 1557 (Brain 
Resource Ltd publishers 2010). 

Face validation 

To achieve face validity, the phrasing of the 45 items was 
slightly adjusted so that the tense and format of the ques- 
tions in each item were consistent, without affecting the con- 
tent of the question. The questions are listed in Appendix 1. 

Construct validation using self-report measures of 
regulation 

Sample: A community sample of 55 healthy volunteers par- 
ticipated in this component of the study (mean age 
29.13 ± 8.80 years, range: 19-55 years; 76.4% female). 
Exclusion criteria were Axis 1 criteria for psychiatric disor- 
der (assessed using the Somatic and Psychological Health 
Report Questionnaire, SPHERES-12; Hickie et al. 2001), 
Patient Health Questionnaire for indicators of eating disor- 
der (Kroenke et al. 2010), indicators of neurological disor- 
der (assessed using items from the mental status 
examination; Trzepacz and Baker 1993), and indicators of 
alcohol and/or substance dependence assessed using the 
AUDIT and Fagerstrom nicotine dependency questionnaire 
(Heatherton et al. 1991; Bush et al. 1998). 

Assessments: 

• Emotion Regulation Scale (Gross and John 2003): To 
assess the capacity to regulate one's emotions in 
terms of both reappraisal and suppression strategies 

• Internal Control Index (ICI; Duttweiler 1984): To 
assess internal locus of control related to the belief 
that reinforcement is contingent on one's own 
behavior, related to self-confidence and autonomy 

Validation outcomes: We conducted correlation analyses 
between the BRISC scales and the ERQ and ICI, using a 
corrected P-value of 0.005. Results demonstrated conver- 
gent construct validation for each BRISC scale as follows: 

• Negative correlations between lower negativity-posi- 
tivity bias and higher scores on ICI components of 
internal control (r = -0.51, P < 0.0001) and self- 
confidence (r = -0.51, P < 0.0001) 

• Positive correlations between higher emotional resil- 
ience and higher scores on the ICI components of 
internal control (r = 0.39, P = 0.003) and self-con- 
fidence (r = 0.39, P = 0.003) 

• Positive correlations between higher social skills and 
higher scores on the ICI perceived control compo- 



L. M. Williams et al. 

nent (r = 0.40, P = 0.003) and the ERQ reappraisal 
strategy component (r = 0.56, P < 0.0001) 

Construct validation of the negativity bias 
measures using genetic, autonomic, and brain 
imaging measures 

Sample: Three hundred and three healthy volunteers of 
European ancestry (mean age 32.92 ± 10.73 years, range: 
18-54 years, 49.5% female) took part and completed the 
BRISC, heart rate recording, and genotyping (Williams 
et al. 2009, 2010). Of these, matched subsets of 39 and 
46 also completed functional magnetic resonance imaging 
(fMRI) (Williams et al. 2009, 2010). 

Assessments: Heart rate and fMRI were recorded during a 
facial emotion viewing task, under both conscious and 
nonconscious conditions. DNA was extracted from cheek 
swab samples and genotyped for the polymorphism of the 
serotonin transporter gene (5-HTT-LPR) and COMT 
Vall08/158Met genotypes (for details of Methods, 
Williams et al. 2009, 2010). 

Validation outcomes: Greater fear reactivity indexed by 
both heart rate and activation of brainstem, amygdala, 
and medial prefrontal cortex circuitry was associated with 
greater negativity relative to positivity bias. This associa- 
tion was pronounced in individuals with the "risk" alleles, 
5 HTT-LPR Short and COMT Met. The findings indicate 
that a higher negativity bias is underpinned by genetic 
and fear circuitry susceptibility for emotional disorder. 

Ecological validation with real-world functional 
capacities 

The association between the full BRISC and proxy mea- 
sures of real-world functional outcomes was established 
in the same sample of 55 participants used to assess con- 
struct validation against the ERQ and IC. These proxy 
measures included the following: 

• Quality of Life assessed by the World Health Organiza- 
tions Qualify of Life scale, brief version (WHOQOL- 
BREF) scale (World Health Organization Group 1998) 

• Satisfaction with Life Scale (SWLS; Diener et al. 
1985) 

• Work productivity, in terms of both absenteeism 
(hours absent from work) and presenteeism (perfor- 
mance level), assessed using the Health and Work Per- 
formance Questionnaire (HPQ; Kessler et al. 2003) 

Validation outcomes: Correlation analyses between the 
BRISC scales and the WHOQOL-BREF, SWLS, and HPQ, 
at a corrected P- value of 0.01, demonstrated the following 
associations: 
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• Negative correlations between lower negativity- 
positivity bias and higher WHOQOL-BREF 
psychological (r = -0.50, P < 0.0001) and social 
relationships (P = —0.39, P = 0.003) components of 
quality of life, and the presenteeism component of 
the productivity on the HPQ (r = -0.39, P = 0.01) 

• Positive correlations between higher emotional resil- 
ience and higher scores on the WHOQOL-BREF psy- 
chological component (r = 0.52, P < 0.0001) and 
satisfaction with life on the SWLS (r = 0.34, 
P = 0.01) 

• Positive correlations between higher social skills and 
higher scores on the WHOQOL-BREF components 
of physical health (r = 0.45, P = 0.001) and envi- 
ronment (r = 0.56, P < 0.0001), satisfaction with 
life (r = 0.42, P = 0.001) and presenteeism on the 
HPQ (r = 0.37, P = 0.008) 

Study research sites 

Participants were recruited from 12 medical research or 
clinical research sites. These sites agreed to collaborate as 
partners with Brain Resource to evaluate brain health in 
patients using a standardized set of assessments and con- 
tribute the data to a centralized library (the Brain 
Resource International Database). The medical research 
sites were located in universities with teaching hospital 
outpatient clinics in psychiatry and psychology. The clini- 
cal research sites were multidisciplinary outpatient clinics 
that offer brain health assessment and treatment services 
(such as EEG testing) for any medical condition. Expert 
clinicians at each site completed diagnostic interviews and 
were blinded to the results of the BRISC and other self- 
report assessments. 

Recruitment 

This retrospective study recruited participants through 
advertising and self-referral. Inclusion criteria were in 
regard to the capacity to undergo a computerized test: 
reading at Year 5 level (equivalent to Year 6 in England 
and fifth grade in the United States), normal (or cor- 
rected to normal) vision, and ability to use a keyboard. 
The protocol received independent ethics committee or 
institutional review board approval before recruitment of 
participants. All participants signed and dated an 
approved informed consent form. Where participants 
consented, these data have also been made available for 
open sharing and secondary analysis by the research com- 
munity (Gordon et al. 2005, 2008). All research is in 
compliance with the Code of Ethics of the World Medical 
Association (Declaration of Helsinki). 
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Main measures 

The assessment of behavioral health status 

At the testing site, participants first completed a computer 
battery of detailed questions to provide an independent 
determination of behavioral health status. This assessment 
comprised established items to assess current or lifetime 
psychiatric and neurological conditions (Table 1). Step- 
wise stratification logic was used to determine "clinical" 
versus "healthy" behavioral health according to the crite- 
ria summarized in Figure 1. 

The BRISC 

After the assessment of behavioral health status, yet in the 
same testing session, participants completed the 45- 
question BRISC (Appendix 1) via computer, which took 
about 10 min to complete. The results provided one score 
for risk (negativity bias) and two scores for coping 
(emotional resilience and social skills; Rowe et al. 2007; 
Williams et al. 2008). As indicated in Appendix 1, the 15- 
question mini version of the BRISC is made up of the five 
highest-loading BRISC items for each of the core content 
domains: negativity bias, emotional resilience, and social 
skills. 

Responses to each BRISC question were made on a scale 
of 1-5, with 5 representing higher functioning (less risk, 
better coping). We summed the responses for negativity 
bias, for emotional resilience, and for social skills (raw 
scores are shown in Appendix 2 for the 45-question BRISC 
and Appendix 3 for the mini-BRISC). These summed 
responses were converted to standardized z-scores, using 
norms in 1317 nonclinical participants established for the 
BRISC (Rowe et al. 2007). The results of both the assess- 
ment of health status and the BRISC were not provided to 
the participant or the investigator at the time of testing. 

Diagnostic interview 

The clinicians at each site also completed a semistruc- 
tured diagnostic interview for each participant which 
included the current status of any psychiatric, psycho- 
logical, or neurological disorder. The interview provided 
confirmation of the disorder against diagnostic criteria, as 
well as the nature of the primary diagnosis. Clinics were 
psychiatrists, neurologists, and clinical psychologists. 

Methods of analysis 

Analyses were undertaken using z-scores for negativity 
bias, emotional resilience, and social skills for the full 
BRISC and the mini-BRISC. Pearson correlations were 
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Table 1. Summary items used in the independent assessment of clinical versus healthy status. 



Number and 
type of 
items* 



Areas assessed 



Source of items 



2 Trigger 
items (yes, 
no) 

8 Follow-up 
12 items 



2 Trigger 
items (yes, 
no) 

8 Follow-up 
1 Trigger item 
(yes, no) 
5 Follow-up 

3 Trigger 
items (yes, 
no) 

16 Follow- 
up 

1 Trigger item 

(yes, no) 

7 Follow-up 
3 Trigger 

items (yes, 

no) 

21 Follow- 
up 

1 Trigger item 
(yes, no) 
30 Follow- 
up 

1 Trigger item 
(yes, no) 
3 Follow-up 

1 Trigger item 
(yes, no) 
7 Follow-up 

1 Trigger item 
(yes, no) 
12 Follow- 
up 



Current or lifetime diagnosis of psychiatric and psychological disorder 
If yes, the nature of the disorder, whether it is current, what the 
previous history is, duration, and treatment 

Current Axis 1 criteria for common mental disorders, focused on 
depressive and anxiety disorders 

Current or lifetime diagnosis of neurological disorder 
If yes, the nature of the disorder, whether it is current, what the 
previous history is, duration, and treatment 

Experience of learning disorder/dyslexia 

If yes, extent of disruption at school, and whether or not dyslexia 

diagnosis was given 
Sleep impairment (past month) 

If yes, extent of impairment and criteria for sleep apnea 



Regular disordered eating 
If yes, items to cover criteria for anorexia and bulimia nervosa 

Regular use of alcohol or other recreational drugs of dependence and 
current medication 

If yes, AUDIT to assess alcohol dependence, Fagerstrom Test for 
Nicotine Dependence and WHO criteria for drug dependence for 
marijuana and other major categories of illicit drugs 
Psychological trauma 
If yes, items for DSM-IV criterion A stressors for PTSD 



Major surgery to brain or spine 
If yes, nature of surgery 

Physical trauma 

If yes, nature of physical injury causing substantive loss of 

consciousness 
Current medication 

If yes, type, reason, dose, frequency for up to three medications 



Mental status examination 
(Trzepacz and Baker 1993) 



Somatic and Psychological Health Report 
Questionnaire (SPHERE-12) screening tool (Hickie 
et al. 2001) 

Mental status examination 



Mental status examination 



Mental status examination 
Maislin Sleep Apnea Index (Maislin et al. 1995) 



Mental status examination 
Patient Health Questionnaire (Spitzer et al. 1999) 

Mental status examination 
AUDIT (Bush et al. 1998) 
Fagerstrom Test for Nicotine Dependence 
(Heatherton et al. 1991) 

Mental status examination 
DSM-IV items for criterion A stressors (Breslau and 
Kessler 2001) 

Mental status examination 



Mental status examination 



Mental status examination 



"These items are implemented in web questionnaire called "WebQ" 



used to examine associations between the three BRISC 
core content domain scores. 

Receiver operating characteristic (ROC) curves were 
then generated using the "Epi" package from the statisti- 
cal analysis program "R" version 2.10.1 (http://www.r- 
project.org/; Ihaka and Gentleman 1996). The goal of the 
ROC curves was to identify the optimal z-score cutpoint 
at which BRISC scores classified participants who were 
independently identified as positive for one or more 



psychiatric-neurological disorders (clinical) versus those 
identified as negative for these disorders (healthy). The 
optimal cutpoint was determined algorithmically to maxi- 
mize sensitivity plus specificity. This threshold was anno- 
tated on these curves with a summary of classification 
performance. A priori z-score thresholds of —0.5, —1.0, 
— 1.5, and —2.0 were also marked on each ROC curve to 
provide a context for the interpretation of the optimal 
threshold. The area under the curve (AUC) statistic was 
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Table 2. Correlations between scores on the 45-question BRISC and 
15-question mini-BRISC* 



YES to 1 + trigger questions 

on current/lifetime 
psychiatric, psychological 
or neurological conditions 



No to all trigger questions 

on current/lifetime 
psychiatric, psychological 
or neurological conditions 



YES to follow- 
up, confirming 
condition is 



NO to follow- 
up, condition 
not present 




Follow-up 



Clinical interview 
confirming diagnostic 




NO 

to additional* 




•Additional trigger questions are on learning disorder, sleep 
disorder, substance dependence, eating disorder, 
psychological trauma, and physical traumatic brain injury. 

Figure 1. Summary of the criteria for independent classification of 
"good" versus "poor" brain health status. 

also generated in each case, where 1.0 is the maximum 
possible value. Sensitivity, specificity, positive predictive 
power, and negative predictive power were tabulated for 
the results at the optimal and a priori z-score thresholds. 

Results 

Characteristics of sample 

From March 2005 through December 2009, 1079 partici- 
pants (mean age = 37.0 years; range: 18-60 years, 51.8% 
female) completed the assessment of behavioral health 
status, the full 45-question BRISC, and the clinician- 
administered diagnostic interview. This sample repre- 
sented a dataset without missing or indeterminate data. 
Overall, 644 participants were identified as being of 
"healthy" status as they answered "no" to all trigger ques- 
tions. The remaining 435 participants were identified as 
being of "clinical" status as they answered "yes" to one or 
more of the trigger questions. The clinical diagnostic 
interview confirmed that all 435 met diagnostic criteria 
for a primary psychiatric, psychological, or neurological 
disorder. Of these 435, 260 met criteria for a primary 
depressive or anxiety disorder, including major depressive 
disorder (128, 29.4%), posttraumatic stress disorder (79, 



Samples 



Negativity Emotional Social 

bias resilience skills 



45-Question BRISC 
Total sample (n = 1079) 
Negativity bias 1.000 
Emotional 
resilience 
Social skills 
Clinical status (n = 435) 
Negativity bias 1.000 
Emotional 
resilience 
Social skills 
Healthy status (n = 644) 
Negativity bias 1.000 
Emotional 
resilience 
Social skills 
15-Question BRISC 
Total sample (n = 1079) 
Negativity bias 1 .000 
Emotional 
resilience 
Social skills 
Clinical status (n = 435) 
Negativity bias 1.000 
Emotional 
resilience 
Social skills 
Healthy status (n = 644) 
Negativity bias 1.000 
Emotional 
resilience 
Social skills 



-.499+ 
1.000 



-.522 t 
1.000 



-.400+ 
1.000 



-.330 1 " 
1.000 



-.341 t 
1.000 



-.141 T 
1.000 



-.297 f 
.312+ 

1.000 

-.316 f 
.375* 

1.000 

— . 1 31 f 
.242 1 " 

1.000 



-.104 f 
.209 1 " 

1.000 

-.1 1 5 1 " 
.239 f 

1.000 

.072 
.169 t 

1.000 



"Correlations calculated the Pearson correlation coefficient. 
f P< 0.0001. 



18.2%), and panic disorder (53, 12.2%). Other disorders 
were traumatic brain injury (86, 19.8%), mild cognitive 
impairment (48, 11.0%), and psychosis (41, 9.4%; speci- 
fied as first onset by clinicians based on no prior episodes 
and being within 3 months of first contact with the 
health service). 

Full BRISC 

In the total sample (« = 1079), negativity-positivity bias 
scores correlated negatively and significantly with both 
emotional resilience (r = —0.499; P < 0.0001) and social 
skills (r = -0.279; P < 0.0001; Table 2). These correla- 
tions are consistent with the theoretical basis of the 
BRISC: that the marker of risk (negativity bias) will be 
inversely related to markers of coping (emotional resil- 
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Breakdown by diagnosis 
n = 1072 (18+ years) 

Abbreviations MCI: Mild Cognitive Impairment, PTSD: Post- 
Traumatic Stress Disorder, ROC: Receiver Operating 
Characteristics 



Figure 2. 45-ltem BRISC. Breakdown of classification by diagnosis for 
negativity bias using the ROC determined threshold. 



ience and social skills). Emotional resilience and social 
skills were found to have a significant overlap (r = 0.312; 
P < 0.0001). The degree of overlap is consistent with 
these markers, reflecting partially separable types of pro- 
tective factors. 

ROC analyses 

In ROC analyses, negativity bias made the largest contri- 
bution to classification. Figure 2 shows the breakdown of 
clinically confirmed diagnoses for negativity bias in the 
"clinical" group. Sensitivity of the BRISC was highest for 
depression, posttraumatic stress disorder, and panic disor- 
der, followed by psychosis, brain injury, and mild cogni- 
tive impairment. 

Table 3 shows the ROC curve analysis results across 
negativity bias, emotional resilience, social skills, and 
combined total scores for the 45-item BRISC. 

For the negativity bias score, the optimal z-score 
threshold for distinguishing clinical status was —1.14. 
This threshold was both sensitive (84.9%) and specific 
(87.6%) in classifying the clinical versus healthy groups. 



Table 3. Summary of sensitivity, specificity, and positive and negative predictive power of the 45-question BRISC scores at z-score thresholds 
of -2, —1.5, — 1, and -0.5 and ROC determined optimal score. 



BRISC scores 



z-Score thresholds 



Negativity bias 
















-2SD threshold 


-1.5SD threshold 


-1SD threshold 


-0.5SD threshold 


-1.14SD 


ROC threshold 


Sensitivity (%) 


91.4 


86.4 


78.9 


67.4 


84.9 




Specificity (%) 


71.9 


84.9 


91.9 


97.4 


87.6 




Positive predictive power (%) 


53.3 


66.8 


77.5 


90.0 


70.7 




Negative predictive power (%) 


96.0 


94.7 


92.5 


89.5 


94.3 




Emotional resilience 
















-2SD threshold 


-1.5SD threshold 


-1SD threshold 


-0.5SD threshold 


-0.43SD 


ROC threshold 


Sensitivity (%) 


64.8 


52.4 


41.3 


28.1 


69.3 




Specificity (%) 


72.4 


85.5 


92.7 


96.9 


70.0 




Positive predictive power (%) 


54.5 


64.9 


74.2 


82.4 


54.1 




Negative predictive power (%) 


80.1 


77.9 


75.6 


72.5 


81.7 




Social skills 
















-2SD threshold 


-1.5SD threshold 


-1SD threshold 


-0.5SD threshold 


-0.50SD 


ROC threshold 


Sensitivity (%) 


54.6 


36.8 


22.5 


15.7 


54.6 




Specificity (%) 


68.1 


83.2 


94.1 


97.9 


68.1 




Positive predictive power (%) 


37.7 


43.6 


57.3 


72.1 


37.7 




Negative predictive power (%) 


80.9 


78.8 


77.4 


76.7 


80.9 




Combined score 
















-2SD threshold 


-1.5SD threshold 


-1SD threshold 


-0.5SD threshold 


-1.57SD 


ROC threshold 


Sensitivity (%) 


69.6 


81.6 


87.2 


92.0 


81.2 




Specificity (%) 


96.3 


91.9 


83.6 


70.7 


92.7 




Positive predictive power (%) 


87.4 


78.7 


66.0 


53.4 


80.2 




Negative predictive power (%) 


89.6 


93.1 


94.6 


96.0 


93.1 





z-Score thresholds are expressed in standard deviations (SD). Results are reported for scores on negativity bias, emotional resilience, social skills, 
and combined total. 
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In addition to good positive predictive power at this 
threshold (70.7%), there was also high negative predictive 
power (94.3%; Table 3). The AUC value of 0.92 indicated 
a very high discrimination, reflective of overall accuracy. 

Emotional resilience scores revealed a lower optimal 
threshold of z = —0.43 for distinguishing clinical from 
healthy status. Sensitivity was at 69.3% and specificity was 
at 70.0%. The results suggested that these scores contrib- 
ute most to negative predictive power (81.7%) for sup- 
porting decisions about confirming good emotional 
health (Table 3). Overall accuracy was high (AUC was 
0.75). 

Social skills scores had an optimal threshold of 
z = —0.50 for classifying clinical from healthy groups. 
Sensitivity was at 54.6% and specificity was at 68.1%. 
Results for these scores suggest that they contribute most 
to negative predictive power (80.9%) relevant to the con- 
firmation of healthy status (Table 3). These scores con- 
tributed to a good overall accuracy (AUC was 0.64). 

When ROC analysis was run for the three BRISC scores 
combined, both positive and negative predictive power 
were maximized (Table 3). The optimal threshold was 
z = —1.57 for the combined scores, with a sensitivity of 



81.2%, specificity of 92.7%, positive predictive power of 
80.2%, and negative predictive power of 93.1%. These 
values generated a high overall accuracy (AUC of 0.93). 

Mini-BRISC 

Correlations for the mini-BRISC showed very nearly the 
same pattern of associations for the total sample, and for 
the clinical and healthy groups, as were found with the 
full BRISC. The only exception was the lack of a signifi- 
cant inverse association between negativity bias and social 
skills for the "clinical" participants (Table 2). 

ROC analyses 

Table 4 summarizes the ROC curve analysis results for 
the 15-item BRISC. The mini-BRISC showed a very simi- 
lar pattern of classification to the full BRISC. For the 
5-item negativity bias score, the optimal threshold was 
z = —1.34, with a sensitivity of 79.9%, specificity of 
89.2%, positive predictive power of 72.2%, and negative 
predictive power of 92.7% (Table 4). Overall accuracy 
remained very high (AUC of 0.92). 



Table 4. Summary of sensitivity, specificity, and positive and negative predictive power of the 1 5-question mini-BRISC scores at z-score thresholds 
of -2, —1.5, — 1, and -0.5 and ROC determined optimal score. 



BRISC scores 


z-Score thresholds 










Negativity bias 














-2SD threshold 


-1.5SD threshold 


-1SD threshold 


-0.5SD threshold 


-1.34SD ROC threshold 


Sensitivity (%) 


89.6 


84.9 


77.1 


69.2 


79.9 


Specificity (%) 


71.6 


83.6 


90.7 


94.7 


89.2 


Positive predictive power (%) 


52.6 


64.6 


74.4 


82.1 


72.2 


Negative predictive power (%) 


95.1 


94.0 


91.8 


89.7 


92.7 


Emotional resilience 














-2SD threshold 


-1.5SD threshold 


-1SD threshold 


-0.5SD threshold 


-0.95SD ROC threshold 


Sensitivity (%) 


9.3 


21.4 


36.8 


53.9 


47.4 


Specificity (%) 


98.7 


96.2 


88.6 


72.2 


83.3 


Positive predictive power (%) 


72.2 


66.7 


53.4 


40.7 


51.1 


Negative predictive power (%) 


75.5 


77.6 


79.9 


81.6 


81.2 


Social skills 














-2SD threshold 


-1.5SD threshold 


-1SD threshold 


-0.5SD threshold 


-0.61 SD ROC threshold 


Sensitivity (%) 


44.6 


27.5 


17.5 


12.1 


43.6 


Specificity (%) 


68.8 


85.5 


94.3 


97.1 


71.1 


Positive predictive power (%) 


33.6 


40.1 


52.1 


59.6 


34.8 


Negative predictive power (%) 


77.9 


76.9 


76.4 


75.8 


78.7 


Combined score 














-2SD threshold 


-1.5SD threshold 


-1SD threshold 


-0.5SD threshold 


-1.31SD ROC threshold 


Sensitivity (%) 


69.2 


77.2 


83.6 


89.2 


80.0 


Specificity (%) 


94.7 


91.0 


84.1 


73.2 


89.3 


Positive predictive power (%) 


82.7 


75.9 


65.9 


54.9 


73.3 


Negative predictive power (%) 


89.3 


91.6 


93.3 


94.8 


92.4 



z-Score thresholds are expressed in standard deviations (SD). Results are reported for scores on negativity bias, emotional resilience, social skills, 
and combined total. 
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The 5-item emotional resilience score showed an optimal 
threshold of z = —0.95. The results suggested that this 
score contributes most to specificity (83.3%) and negative 
predictive power (81.2%) for supporting decisions about 
confirming healthy status, rather than sensitivity to a clini- 
cal condition (Table 4). Accuracy was retained at a simi- 
larly high level to that for the full BRISC (AUC of 0.69). 

For the 5-item social skills score, the optimal threshold 
was z = —0.61. The results suggest that this score also 
contributes most to specificity (71.1%) and negative pre- 
dictive power (78.7%) for classifying good brain health 
(Table 4). Overall accuracy remained in the moderate to 
high range (AUC of 0.58). 

For the three mini-BRISC scores combined, both posi- 
tive and negative predictive power were maximized, as they 
were for the 45-question version (Table 4). The optimal 
threshold was z = — 1.31 for the combined scores, with a 
sensitivity of 80.0%, specificity of 89.3%, positive predic- 
tive power of 73.3%, and negative predictive power of 
92.4%. Overall accuracy was similarly high (AUC of 0.92). 

Discussion 

This study evaluated the performance of the web- 
delivered BRISC (full and mini versions) in identifying 



emotional dysregulation, a hallmark of clinical status in 
patients with a range of psychiatric and neurological con- 
ditions. The study results were consistent across the full- 
and mini-BRISC versions. For the three BRISC scores 
combined, the full 45-question BRISC had a high overall 
accuracy of 0.93 (Fig. 3). The best classification of clinical 
status was at the threshold of z = —1.57, substantially 
below the population average of 0. The mini 15-question 
BRISC showed a similarly high accuracy of 0.92 (Fig. 4). 
These results support the effectiveness of the BRISC for 
identifying risk for a clinical disorder, manifested as loss 
of emotion regulation. 

Negativity bias scores made the main contribution to 
the determination of clinical versus healthy status. For the 
full 45-question BRISC, the negativity bias score on its 
own detected clinical status best at a z-score of —1.14, 
consistent with a threshold of clinical meaningfulness. At 
this threshold, negativity bias scores showed high accuracy 
for detecting outpatients with a clinical condition. Across 
diagnostic categories, negativity bias scores showed the 
highest detection for major depressive disorder, post- 
traumatic stress disorder, and panic disorder. This profile 
of accuracy was duplicated for the mini version's negativ- 
ity bias scores. 



(a) (b) 




1 -Specificity 1 -Specificity 

Figure 3. Receiver operating curve results for the 45-item BRISC, for negativity bias (a), emotional resilience (b), social skills (c), and all three 
scores combined (d). 
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Emotional resilience and social skills separated clinical 
from healthy status at a higher z-score threshold than did 
negativity bias. Both emotional resilience and social skills 
scores showed high specificity. These scores are consistent 
with the view that a higher-than-average coping capacity 
may offset risk for a clinical condition and thus support 
screening and triaging decisions. Results were duplicated 
for the full and mini version of these scores. 

These findings suggest that the BRISC functions to 
effectively assess the spectrum of poor through to effective 
emotion regulation. It provides a quick and accurate 
screen for identifying risk of a clinical disorder across 
multiple diagnostic categories that takes into account 
both susceptibility and coping factors. These findings sup- 
port the use of the BRISC as an objective pan-diagnostic 
screen for multiple populations, from general through 
specialty. It expands on the current tools that screen for a 
particular diagnosis such as major depressive disorder 
(Mulrow et al. 1995; Rush et al. 2003). The sensitivity of 
the BRISC was highest in participants with diagnoses of 
depressive and anxiety disorders, consistent with the con- 
cept of negativity bias, but also retained a good level of 
classification across the other diagnostic categories. It also 
accomplishes the consideration of coping factors, and 
how they may offset risk factors, which has not been a 
part of previous instruments. 



Strengths of the study include the large sample size, 
and coverage of multiple diagnostic groups. Future 
research is needed to extend the findings and address its 
limitations. The range of clinical participants included in 
the study was defined by the types of clinics being oper- 
ated in participating sites. Future studies are needed to 
extend the evaluation to other diagnostic groups. Valida- 
tion work with the BRISC has shown it correlates with 
real-world capacities such as quality of life and work 
productivity. Here, the cross-sectional design means there 
was no opportunity to follow up participants to assess 
the BRISC in relation to real-world functional outcomes 
over time. A controlled design would be of value, in 
which the BRISC is evaluated pretreatment and post- 
treatment. Future research is also needed to evaluate the 
replicability of the current findings, and their generaliz- 
ability to additional populations. A prospective study 
might address this study's limitations involving the range 
of clinical participants and the lack of participant follow- 
up in relation to outcomes. Another valuable area for 
future studies would be to compare the sensitivity/ 
specificity of the BRISC against multiple disorder-specific 
measures. 

The BRISC offers a web-based tool to support the effi- 
cient management of mental and neurological health 
across populations. Its accuracy enables nonspecialist 






0.4 



0.6 



1 -Specificity 

Figure 4. Receiver operating curve results for the 15-item BRISC, for negativity bias (a), emotional resilience (b), social skills (c), and all three 
scores combined (d). 
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physicians and physician assistants to confidently screen 
for emotion dysregulation, as a core feature of mental 
health issues. The mini-BRISC offers an even briefer 
screen of emotional health that retains high levels of 
accuracy and may be especially suitable when a heavy 
patient load constrains the clinician's time. BRISC scores, 
especially negativity bias, capture maladaptive emotional 
reactivity to daily events and could be used to identify 
this feature of risk for depressive and anxiety disorders 
within other chronic conditions. The coping scores of 
emotional resilience and social skills may help to deter- 
mine which patients are best able to cope with clinical 
issues and engage social support. Using this tool may help 
support early management of emotional mental health 
issues and limit the disproportionate flow on effects to 
disability and loss of productivity. 
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Appendix 1. BRISC items that contribute to negativity bias, 
emotional resilience, and social skills scores. The subset of items that 
define the mini version of the BRISC is indicated by bold text. Reverse 
scored items indicated by (-)*. 



Appendix 1. Continued 
I tried to build my close relationships with people 

I took part in social groups 

People reacted to me as if I would do whatever it takes to get 
ahead (-1) 

I enjoyed socializing and chatting to other people 

I tried out exciting places and things to do 

I was unable to become enthusiastic about anything (-) 

"These items have been trademarked as the "BRISC", by Brain 
Resource, and are available as a web-delivered assessment for clinical 
and research uses. 



Appendix 2. 45-ltem BRISC summary of the raw scores and their 
corresponding z and standardized 10 (STEN) scores for the composite 
markers; negativity bias, emotional resilience, and social skills. Raw 
scores were converted to standardized z-scores using the nonclinical 
norm sample of n = 1317*. 



Negativity bias Emotional resilience 

(20 items) (14 items) Social skills (1 1 items) 



Raw 




Raw 


Z ~ 




Raw 


Z ~ 






S r 


_) i r_i n it.ui tr 




STEN 






STEN 


20 


-6.54 


1 14 


-6.11 


1 


11 


-6.16 


1 


21 


-6.44 


1 15 


-5.95 


1 


12 


-5.95 


1 


22 


-6.33 


1 16 


-5.78 


1 


13 


-5.74 


1 


23 


-6.22 


1 17 


-5.61 


1 


14 


-5.53 


1 


24 


-6.11 


1 18 


-5.45 




15 


-5.32 




25 


-6.01 


1 19 


-5.28 




16 


-5.11 




26 


-5.90 


1 20 


-5.12 




17 


-4.90 




27 


-5.79 


1 21 


-4.95 




18 


-4.69 




28 


-5.69 


1 22 


-4.79 




19 


-4.47 




29 


-5.58 


1 23 


-4.62 




20 


-4.26 




30 


-5.47 


1 24 


-4.46 




21 


-4.05 




31 


-5.36 


1 25 


-4.29 




22 


-3.84 




32 


-5.26 


1 26 


-4.13 




23 


-3.63 




33 


-5.15 


1 27 


-3.96 




24 


-3.42 




34 


-5.04 


1 28 


-3.79 




25 


-3.21 




35 


-4.94 


1 29 


-3.63 




26 


-3.00 




36 


-4.83 


1 30 


-3.46 




27 


-2.79 




37 


-4.72 


1 31 


-3.30 




28 


-2.58 




38 


-4.62 


1 32 


-3.13 




29 


-2.37 




39 


-4.51 


1 33 


-2.97 




30 


-2.16 


1.2 


40 


-4.40 


1 34 


-2.80 




31 


-1.95 


1.6 


41 


-4.29 


1 35 


-2.64 




32 


-1.74 


2.0 


42 


-4.19 


1 36 


-2.47 




33 


-1.53 


2.4 


43 


-4.08 


1 37 


-2.31 




34 


-1.32 


2.9 


44 


-3.97 


1 38 


-2.14 


1.2 


35 


-1.11 


3.3 


45 


-3.87 


1 39 


-1.97 


1.6 


36 


-0.90 


3.7 


46 


-3.76 


1 40 


-1.81 


1.9 


37 


-0.69 


4.1 


47 


-3.65 


1 41 


-1.64 


2.2 


38 
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{Continued) 



Negativity bias 
I was often stressed, with my nerves on edge 

I felt I have no value as a person 
I often felt annoyed at the way people treated me 
I lost hope and wanted to give up when something went 
wrong 

I felt out of control of my life and needed others to help me 
I was reliable and could be counted on to keep my word (-) 
I found it hard to wind down 
I was jumpy and agitated 
I found it difficult to relax 

I was intolerant of anything that kept me from getting on with 

things 
I was rather touchy 

I tended to over-react to situations 

I felt that I had nothing to look forward to 
I felt that life was meaningless 
I felt down-hearted and blue 
I felt I wasn't worth anything 

I couldn't seem to experience any positive feeling at all 
I found it difficult to work up the initiative to do things 
I felt I close to panic 

I was worried about situations in which I might panic and make a 

fool of myself 
Emotional resilience 
I felt very satisfied with the way I look and act 
I responded best to positive feedback about myself 
When receiving negative comments about myself, I looked 

for positive things to counter balance those comments 
I was often irritable and argued with people around me (-) 
I was able to plan ahead to meet deadlines 
I was proud of myself and some people might have thought I 

was putting myself first (-) 
I was conscientious about everything I was asked to do 
I was organized and could work toward a goal in a step by step way 
I usually dithered around before I getting down to focusing on a 

task (-) 

I worked hard, and always tried to achieve my goals 
There were times when people couldn't rely on me as 
much as they should have be able to (-) 

I usually tried to consider other people's needs and feelings 
I was always successful at completing my tasks, even if I 
had more tasks than others 

I was always disorganized and in a mess (-) 
Social skills 

I could sense the mood of a group and discuss unspoken 
feelings 

I always tried to put myself into the place of those I was talking with 
I got feedback that I am a sensitive and understanding 
person 

I usually took the initiative and introduced myself to 
strangers 

I got enormous satisfaction by getting people to 
like me 



(Continued) 
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Appendix 2. Continued 



Negativity bias Emotional resilience 
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*0f these nonclinical norms, 579 were also included in this study and 
in each case were identified as having good brain health status. 



BRISC Screen for Emotional Health 

Appendix 3. 15-item BRISC summary of the raw scores and their 
corresponding z and standardized 10 (STEN) scores for the composite 
markers; negativity bias, emotional resilience, and social skills. Raw 
scores were converted to standardized z-scores using the nonclinical 
norm sample of n = 1317*. 
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*Of these nonclinical norms, 579 were also included in this study and 
in each case were identified as having good brain health status. 
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